Video Instance Segmentation by Instance Flow Assembly

Instance segmentation is a challenging task aiming at classifying and segmenting all object instances of specific classes. While two-stage box-based methods achieve top performances in the image domain, they cannot easily extend their superiority into video domain. This because usually deal with features or images cropped from detected bounding boxes without alignment, failing to capture pixel-level temporal consistency. We embrace observation that bottom-up dealing box-free could offer accurate spacial correlations across frames, which can be fully utilized for pixel level tracking. first propose our framework equipped context fusion module better encode inter-frame correlations. Intra-frame cues semantic localization are simultaneously extracted reconstructed by corresponding decoders after shared backbone. For efficient robust tracking among instances, we introduce an instance-level correspondence adjacent represented center-to-center flow, termed as instance assemble messy dense correspondences. Experiments demonstrate proposed method outperforms state-of-the-art online (taking image-level input) on Youtube-VIS dataset [46].